ABNORMAL PATTERN CANDIDATE DETECTION PROCESSING 
METHOD AND SYSTEM 



BACKGROUND OF THE INVENTION 
Field of the Invention 

This invention relates to an abnormal pattern 
candidate detection processing method and system. This 
invention particularly relates to an improvement in information 
output ted by image output means constituting the abnormal 
pattern candidate detection processing system. 

Description of the Related Art 

In the field of medicine, various kinds of image 
forming modalities (i.e., image input apparatuses), such as 
computed tomography (CT) scanners, magnetic resonance imaging 
(MRI) apparatuses, and computed radiography (CR) apparatuses, 
have become popular as apparatuses for forming images to be 
used in making a diagnosis. Also, abnormal pattern candidate 
detection processing systems (computer aided medical image 
diagnosing systems) have heretofore been proposed, wherein a 
candidate for an abnormal pattern embedded in an image of an 
object, which has been acquired with one of the image forming 
modalities, is detected automatically by the utilization of 
a computer from an image signal representing the object. The 
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abnormal pattern candidate detection processing systems are 
described in, for example, Patent Document 1. 

The primary objective of the proposed abnormal 
pattern candidate detection processing systems primarily is 
to detect an abnormal pattern (i.e., a tumor pattern, a 
microcalcif ication pattern, or the like) , which suggests the 
presence of breast cancer, or the like, from a mammogram. The 
abnormal pattern candidate detection processing systems are 
provided with abnormal pattern candidate detecting means 
utilizing an iris filter, wherein image density gradients (or 
luminance gradients ) in an image are represented by image density 
gradient vectors, and an image area, which is associated with 
a high degree of convergence of the image density gradient 
vectors, is detected as an abnormal pattern candidate. 
Alternatively, the abnormal pattern candidate detection 
processing systems are provided with abnormal pattern candidate 
detecting means utilizing a morphological filter, wherein a 
multi- structure element in accordance with the size of an 
abnormal pattern to be detected is utilized, and an image area, 
at which the image density changes in a range spatially narrower 
than the multi- structure element, is detected as an abnormal 
pattern candidate . 
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With the abnormal pattern candidate detecting means 
utilizing the iris filter, a candidate for a tumor pattern (a 
form of the abnormal pattern) , which is a form of breast cancer, 
or the like, is capable of being detected automatically. With 
the abnormal pattern candidate detecting means utilizing the 
morphological filter, a candidate for a microcalcif ication 
pattern (a form of the abnormal pattern) , which is a different 
form of breast cancer, or the like, is capable of being detected 
automatically . 

In the abnormal pattern candidate detection 
processing systems, when an image signal representing an image 
(i.e. , an original image, such as a mammogram) to be subjected 
to abnormal pattern candidate detection processing is received, 
the abnormal pattern candidate detecting means described above 
detects an abnormal pattern candidate. Then, a 

region-of -interest setting means sets a local area limited 
region, which contains the detected abnormal pattern candidate 
and a neighboring region, as a region of interest (hereinbelow 
referred to as the ROI) . Also, local area limited image 
processing means performs specified image processing, such as 
enhancement processing in accordance with an attribute of the 
detected abnormal pattern candidate (i.e. , whether the detected 
abnormal pattern candidate is a tumor pattern or a 
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microcalcif ication pattern) , on the ROI image . Further, entire 
area image processing means performs predetermined image 
processing on an entire area image representing the original 
image, such that a visible image may be obtained. The visible 
image has good image quality and is capable of serving as an 
effective tool in, particularly, the efficient and accurate 
diagnosis of an illness. Furthermore, layout image forming 
means forms a single layout image from the entire area image, 
whichhas been obtained from the predetermined image processing, 
and the ROI image, which has been obtained from the specified 
image processing, in accordance with a layout having been set 
previously. An image signal (hereinbelow referred to as the 
layout image signal) representing the layout image is output 
to image display means, such as a cathode ray tube (CRT) display 
device, or image printing means, such as a laser printer (LP) . 

Figure 21 is an explanatory view showing an example 
of outputting from an abnormal pattern candidate detection 
processing system to a CRT display device. Specifically, in 
the case of mammography, for example, as illustrated in Figure 
21, an image P of the right breast of a patient and an arrow 
mark Mr , which indicates an abnormal pattern candidate PI having 
been detected, may be superposed on each other and displayed 
on the right half of a display screen of the CRT display device . 
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Also, an image P 1 of the left breast of the patient, from which 
no abnormal pattern has been detected, may be displayed on the 
left half of the display screen. A layout image L, which has 
been laid out in this manner, is outputted to the CRT display 
device . 

As described above, with the abnormal pattern 
candidate detection processing systems, the features, 
primarily shape or form features, and the like, of the abnormal 
pattern are ultimately expressed numerically. The detection 
processing level for the abnormal pattern candidate is 
determined with statistical techniques by use of the features 
of the abnormal pattern. Also, the abnormal pattern candidate 
is detected in accordance with the thus determined detection 
processing level. In this manner, the abnormal pattern 
candidate is capable of being detected uniformly regardless 
of the diagnostic experience or the diagnostic skill of an 
operator. 

The pattern, which is detected by the abnormal pattern 
candidate detection processing systems, is the candidate for 
the abnormal pattern. An operator, such as a medical doctor, 
ultimately makes a judgment as to whether the abnormal pattern 
■s candidate having been detected by the abnormal pattern candidate 

detection processing systems is a true abnormal pattern. The 
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operator, such as the medical doctor, makes the judgment by- 
viewing the image of the abnormal pattern candidate, which image 
is outputted by image displaying means, such as a CRT display 
device, or printing means. Therefore, it is desired that 
various kinds of information useful for making the final judgment 
be furnished to the operator who views the image of the abnormal 
pattern candidate . 

Accordingly, the applicant proposed an abnormal 
pattern candidate detection processing system, wherein an index 
value, which concerns a judgment as to whether a pattern is 
to be detected as an abnormal pattern candidate in abnormal 
pattern candidate detection processing, a value representing 
an allowance of detection with respect to an image signal , which 
has been detected as an image signal representing an abnormal 
pattern candidate, and the like, are displayed together with 
the image of the detected abnormal pattern candidate. The 
proposed abnormal pattern candidate detection processing 
system is described in Patent Document 2 shown below. 

Patent Document 1 : 

U.S. Patent No. 5,761,334 

Patent Document 2 : 

Japanese Unexamined Patent Publication 
No. 2001-299740 
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However, the index value displayed by the proposed 
abnormal pattern candidate detection processing system is 
merely an index value regarded as being theoretically effective , 
and is not an index value reflecting a correlation between the 
index value and possibility of a pattern being a malignant 
pattern. Therefore, if the operator who views the image of 
the abnormal pattern candidate does not know the correlation 
described above through experience, the operator who views the 
image of the abnormal pattern candidate cannot utilize the 
displayed index value as information useful for making a 
diagnosis. Also, in cases where the operator who views the 
image of the abnormal pattern candidate knows the correlation 
described above, it is not always possible for the operator 
to intuitively make a judgment as to the possibility of a pattern 
being a malignant pattern, from the displayed index value. 
SUMMARY OF THE INVENTION 

The primary object of the present invention is to 
provide an abnormal pattern candidate detection processing 
method, wherein specific useful information is outputted, such 
that the capability of an image of an abnormal pattern candidate 
serving as an effective tool in, particularly, efficient and 
accurate diagnosis of an illness, may be enhanced even further. 



7 



Another object of the present invention is to provide 
an abnormal pattern candidate detection processing system for 
carrying out the abnormal pattern candidate detection 
processing method. 

The present invention provides a first abnormal 
pattern candidate detection processing method, comprising the 
steps of: 

i) detecting an abnormal pattern candidate, which 
is embedded in a medical image, in accordance with a medical 
image signal representing a medical image, and 

ii) outputting at least information for specifying 
the detected abnormal pattern candidate, 

wherein the method further comprises the step of 
calculating a degree of certainty about malignancy, which 
represents a level of possibility of a pattern being a malignant 
pattern, with respect to the abnormal pattern candidate, the 
calculation being made in accordance with an index value 
representing a feature of the abnormal pattern candidate and 
in accordance with a correlation between the index value and 
possibility of a pattern being a malignant pattern, which has 
been obtained from clinical results, and 

the step of outputting at least the information for 
specifying the detected abnormal pattern candidate is a step 
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of outputting information representing the degree of certainty 
about malignancy with respect to the abnormal pattern candidate 
together with the information for specifying the detected 
abnormal pattern candidate . 

The present invention also provides a first abnormal 
pattern candidate detection processing system for carrying out 
the first abnormal pattern candidate detection processing 
method. Specifically, the present invention also provides a 
first abnormal pattern candidate detection processing system, 
comprising : 

i) abnormal pattern candidate detecting means for 
detecting an abnormal pattern candidate, which is embedded in 
a medical image, in accordance with a medical image signal 
representing a medical image, and 

ii) image output means for outputting at least 
information for specifying the detected abnormal pattern 
candidate, 

wherein the system further comprises malignancy 
certainty degree calculating means for calculating a degree 
of certainty about malignancy, which represents a level of 
possibility of a pattern being a malignant pattern, with respect 
to the abnormal pattern candidate, the calculation being made 
in accordance with an index value representing a feature of 



the abnormal pattern candidate and in accordance with a 
correlation between the index value and possibility of a pattern 
being a malignant pattern, which has been obtained from clinical 
results, and 

the image output means outputs information 
representing the degree of certainty about malignancy with 
respect to the abnormal pattern candidate together with the 
information for specifying the detected abnormal pattern 
candidate . 

The term "index value representing a feature of an 
abnormal pattern candidate" as used herein means at least one 
of various index values , which represent features of the detected 
abnormal pattern candidate. For example, the index value 
representing the feature of the abnormal pattern candidate may 
be at least one from among index values that represent the 
features of the abnormal pattern candidate, such as a shape, 
a size, a density, and a gray level pattern of the abnormal 
pattern candidate. However, the index value representing the 
feature of the abnormal pattern candidate may be any index value , 
whose correlations with the possibility of a pattern being a 
malignant pattern are ultimately obtained. 

Also, the degree of certainty about malignancy is 
calculated in accordance with the "actual correlation" between 
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the index value, which represents the feature of the pattern 
within a certain image region, and the level of possibility 
of the pattern within the image region being a malignant pattern. 
The degree of certainty about malignancy is definitely different 
from an index value employed in conventional techniques, wherein 
the index value associated with a certain image region is 
" regarded" as being a factor reflecting the level of possibility 
that the image region will be a malignant pattern, and the index 
value is utilized directly as the "degree of malignancy. " 

The correlation between the index value and 
possibility of a pattern being a malignant pattern, which has 
been obtained from clinical results, may be, for example, the 
relationship between the level of the index value having been 
obtained from an operation, wherein a calculation of the index 
value of each pattern is made with respect to each of a plurality 
of cases of diseases, and wherein it is actually confirmed with 
pathological examinations and the like whether the state 
associated with the pattern is a malignant state , a benign state , 
or a normal state. In addition, the correlation may be a 
proportion associated with each level of the index value, which 
proportion is occupied by each of the malignant state , the benign 
state, and the normal state. In such cases, for example, the 
degree of certainty about malignancy may be calculated as the 
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proportion associated with the level of the index value of the 
image region subjected to the calculation, which proportion 
is occupied by the malignant state. Specifically, in cases 
where the index value of the image region is 1 , and the proportion 
associated with the index value of 1 which is occupied by the 
malignant state is 7 0%, the degree of certainty about malignancy 
may be interpreted as being 70%. 

In the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the index value may be an index value utilized for 
the detection of the abnormal pattern candidate. 

In cases where the processing for detecting the 
abnormal pattern candidate is the processing for detecting a 
tumor pattern candidate, the index value utilized for the 
detection of the abnormal pattern candidate may be a degree 
of convergence of the tumor pattern, an output value of an iris 
filter, a degree of spread, elongation, roughness of periphery, 
circularity, entropy, or the like. In cases where the 
processing for detecting the abnormal pattern candidate is the 
processing for detecting a microcalcif ication pattern 
candidate, the index value utilized for the detection of the 
abnormal pattern candidate may be the density of 
microcalcif ication patterns, or the like. Also, the index 
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value utilized for the detection of the abnormal pattern 
candidate may be a Mahalanobis distance, which reflects the 
similarity of an image pattern with respect to a malignant 
pattern or a benign pattern, or a likelihood ratio of the 
Mahalanobis distance, which will be described later . Further, 
in cases where an image region, which is associated with the 
index value larger than a threshold value, is to be extracted 
as an abnormal pattern candidate, the index value utilized for 
the detection of the abnormal pattern candidate may be a degree 
of allowance, which represents the extent of allowance of the 
index value with respect to the threshold value. By way of 
example, the degree of allowance may be represented by K/T, 
lOOx (K-T) /K (%) , or (K-T) , where K represents the index value, 
and T represents the threshold value. 

Also, the information for specifying the detected 
abnormal pattern candidate and the information representing 
the degree of certainty about malignancy with respect to the 
abnormal pattern candidate may be a mark, which is displayed 
at a position within the medical image, to indicate the abnormal 
pattern candidate. The kind of the mark may be altered in 
accordance with the degree of certainty about malignancy. The 
mark may be, for example, an "arrow mark" for indicating the 
abnormal pattern candidate, a "closed curve" or a "rectangular 
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frame," which surrounds the abnormal pattern candidate, or a 
"number" displayedby the side of the abnormal pattern candidate . 
The color, the thickness of the line, the number of the lines, 
the value of the number, or the like, may be altered in accordance 
with the degree of certainty about malignancy. 

The present invention further provides a second 
abnormal pattern candidate detection processing method, 
comprising the steps of: 

i) detecting an abnormal pattern candidate, which 
is embedded in a medical image, in accordance with a medical 
image signal representing a medical image, and 

ii) outputting at least information for specifying 
the detected abnormal pattern candidate, 

wherein the method further comprises the steps of: 

a) selecting an arbitrary region in the medical 

image , and 

b) calculating a degree of certainty about 
malignancy, which represents a level of possibility of a pattern 
being a malignant pattern, with respect to a pattern embedded 
in the selected region, the calculation being made in accordance 
with an index value representing a feature of the pattern 
embedded in the selected region and in accordance with a 
correlation between the index value and possibility of a pattern 
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being a malignant pattern, which has been obtained from clinical 
results, and 

the step of outputting at least the information for 
specifying the detected abnormal pattern candidate is a step 
of further outputting information representing the degree of 
certainty about malignancy with respect to the pattern embedded 
in the selected region. 

The present invention still further provides a second 
abnormal pattern candidate detection processing system for 
carrying out the second abnormal pattern candidate detection 
processing method. Specifically, the present invention 
provides a second abnormal pattern candidate detection 
processing system, comprising: 

i) abnormal pattern candidate detecting means for 
detecting an abnormal pattern candidate, which is embedded in 
a medical image, in accordance with a medical image signal 
representing a medical image, and 

ii) image output means for outputting at least 
information for specifying the detected abnormal pattern 
candidate , 

wherein the system further comprises: 

a) region selecting means for selecting an 
arbitrary region in the medical image, and 



b) malignancy certainty degree calculating 
means for calculating a degree of certainty about malignancy, 
which represents a level of possibility of a pattern being a 
malignant pattern, with respect to a pattern embedded in the 
selected region, the calculation being made in accordance with 
an index value representing a feature of the pattern embedded 
in the selected region and in accordance with a correlation 
between the index value and possibility of a pattern being a 
malignant pattern, which has been obtained from clinical results , 
and 

the image output means further outputs information 
representing the degree of certainty about malignancy with 
respect to the pattern embedded in the selected region. 

In the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention and the second abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the information representing the degree of certainty 
about malignancy may be a numerical value. For example, as 
the information representing the degree of certainty about 
malignancy, the possibility of a pattern being a malignant 
pattern may be represented by numerical values ranging from 
0% to 100%. 
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Alternatively, the information representing the 
degree of certainty about malignancy may be a warning message, 
which is altered in accordance with the degree of certainty 
about malignancy. For example, in cases where the degree of 
certainty about malignancy with respect to the detected abnormal 
pattern candidate or the pattern embedded in the selected region 
is comparatively high, a warning message representing "HIGH 
MALIGNANCY" may be displayed in the vicinity of the pattern. 
Alternatively, in such cases, a warning message representing 
"THERE IS A HIGHLY MALIGNANT PATTERN" may be displayed at a 
position spaced from the pattern. In cases where the degree 
of certainty about malignancy with respect to the detected 
abnormal pattern candidate or the pattern embedded in the 
selected region is comparatively low, a message representing 
"LOW MALIGNANCY" may be displayed in the vicinity of the pattern. 
Alternatively, in such cases, a message representing "THERE 
IS NOT A HIGHLY MALIGNANT PATTERN" may be displayed at a position 
spaced from the pattern. 

The present invention also provides a third abnormal 
pattern candidate detection processing method, comprising the 
steps of: 
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i) detecting an abnormal pattern candidate, which 
is embedded in a medical image, in accordance with a medical 
image signal representing a medical image, and 

ii) outputting at least information for specifying 
the detected abnormal pattern candidate, 

wherein the method further comprises the steps of: 

a) calculating a degree of certainty about 
malignancy, which represents a level of possibility of a pattern 
being a malignant pattern , with respect to a predetermined region 
in the medical image, which has been set for each of pixels 
in the medical image , as the degree of certainty about malignancy 
corresponding to each of the pixels in the medical image, the 
calculation being made in accordance with an index value 
representing a feature of apattern embedded in the predetermined 
region and in accordance with a correlation between the index 
value and possibility of a pattern being a malignant pattern, 
which has been obtained from clinical results, and 

b) forming a distribution image signal 
representing a distribution image, which represents a 
distribution of the degrees of certainty about malignancy in 
the medical image , in accordance with the thus calculated degrees 
of certainty about malignancy, each of which corresponds to 
one of the pixels, and 
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the step of outputting at least the information for 
specifying the detected abnormal pattern candidate is a step 
of further outputting the distribution image in accordance with 
the thus formed distribution image signal. 

The present invention further provides a third 
abnormal pattern candidate detection processing system for 
carrying out the third abnormal pattern candidate detection 
processing method. Specifically, the present invention 
provides a third abnormal pattern candidate detection 
processing system, comprising: 

i) abnormal pattern candidate detecting means for 
detecting an abnormal pattern candidate, which is embedded in 
a medical image, in accordance with a medical image signal 
representing a medical image, and 

ii) image output means for outputting at least 
information for specifying the detected abnormal pattern 
candidate , 

wherein the system further comprises: 

a) malignancy certainty degree calculating 
means for calculating a degree of certainty about malignancy, 
which represents a level of possibility of a pattern being a 
malignant pattern, with respect to a predetermined region in 
the medical image, which has been set for each of pixels in 

19 



the medical image, as the degree of certainty about malignancy 
corresponding to each of the pixels in the medical image, the 
calculation being made in accordance with an index value 
represent ing a feature of a pattern embedded in the predetermined 
region and in accordance with a correlation between the index 
value and possibility of a pattern being a malignant pattern, 
which has been obtained from clinical results, and 

b) distribution image signal forming means 
for forming a distribution image signal representing a 
distribution image, which represents a distribution of the 
degrees of certainty about malignancy in the medical image, 
in accordance with the thus calculated degrees of certainty 
about malignancy, each of whichs corresponds to one of the pixels , 
and 

the image output means further outputs the 
distribution image in accordance with the distribution image 
signal, which has been formed by the distribution image signal 
forming means . 

In the third abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the "predetermined region" in the medical image, 
which has been set for each of the pixels in the medical image, 
may be, for example, a circular region having its center at 
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one pixel of interest and having a radius of R. Also, in such 
cases, as for the index value representing the feature of the 
pattern embedded in the predetermined region in the medical 
image, which has been set for each of the pixels in the medical 
image, an output value of an iris filter obtained with respect 
to the circular region acting as the predetermined region may- 
be taken as the "index value" corresponding to the pixel of 
interest . 

In the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the second abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, and the third abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the medical image may be a mammogram. 

Also, as the image output means, image displaying 
means, such as a CRT display device or a liquid crystal monitor, 
or printing means, such as a laser printer, may be employed. 

Further, the information for specifying the detected 
abnormal pattern candidate may be an image representing the 
abnormal pattern candidate. Alternatively, the information 
for specifying the detected abnormal pattern candidate may be 
numerical information, which represents the position, the shape 
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the size, or the like, of the abnormal pattern candidate. In 
cases where the image representing the abnormal pattern 
candidate is employed as the information for specifying the 
detected abnormal pattern candidate, the image representing 
the abnormal pattern candidate may be an image area of the 
abnormal pattern candidate itself (or an image area of the 
abnormal pattern candidate itself having been subj ected to image 
processing, such as image size enlargement or reduction 
processing, processing in the frequency domain, or sharpness 
enhancement processing) . Alternatively, in such cases, the 
image representing the abnormal pattern candidate may be a marker , 
such as an arrow mark, which indicates the abnormal pattern 
candidate in the entire area image, a rectangular or circular 
region- of -interest (ROI) frame, which surrounds the abnormal 
pattern candidate in the entire area image, or a contour frame, 
which extends along the contour of the abnormal pattern candidate 
in the entire area image. 

Iris filtering processing, morphological filtering 
processing, and processing for calculating a likelihood ratio 
of a Mahalanobis distance will be described hereinbelow. 
(Iris filtering processing) 
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The iris filtering processing is suitable for the 
detection of a tumor pattern candidate embedded in a radiation 
image . 

It has been known that, for example, in a radiation 
image recorded on X-ray film (i.e., an image represented by 
an image signal of a high signal level for a high image density) , 
the image density values of a tumor pattern are ordinarily 
slightly smaller than the image density values of the surrounding 
image areas. The image density values of the tumor pattern 
are distributed such that the image density value becomes smaller 
from the periphery of an approximately circular tumor pattern 
toward the center point of the tumor pattern. Thus the 
distribution of the image density values of the tumor pattern 
has gradients of the image density values. Therefore, in the 
tumor pattern, the gradients of the image density values can 
be found in local areas, and the gradient lines (i.e. , gradient 
vectors) converge in the directions heading toward the center 
point of the tumor pattern. 

The iris filter calculates the gradients of image 
signal values , which are representedby the image density values , 
as gradient vectors and outputs information representing the 
degree of convergence of the gradient vectors. With the iris 
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filtering processing, a tumor pattern is detected in accordance 
with the degree of convergence of the gradient vectors. 

Specifically, by way of example, as illustrated in 
Figure 2A, a tumor pattern Pj may be embedded in a mammogram P. 
As illustrated in Figure 2B, the gradient vector at an arbitrary 
pixel in the tumor pattern P a is directed to the vicinity of 
the center point of the tumor pattern P a . On the other hand, 
as illustrated in Figure 2C, in an elongated pattern P K , such 
as a blood vessel pattern or a mammary gland pattern, gradient 
vectors do not converge toward a specific point. Therefore, 
the distributions of the orientations of the gradient vectors 
in local areas may be evaluated, and a region, in which the 
gradient vectors converge toward a specific point, may be 
detected. The thus detected region may be taken as a tumor 
pattern candidate , which is considered as being a tumor pattern . 
As illustrated in Figure 2D, in a pattern P L , in which elongated 
patterns , such as mammary gland patterns , intersect each other, 
gradient vectors are liable to converge toward a specific point . 
Therefore , the pattern P L may be detected falsely as an abnormal 
pattern candidate (a false positive (FP) ) . 

The iris filtering processing is based on the 
fundamental concept described above. The steps of the iris 
filtering processing algorithm will be described hereinbelow. 
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(Step 1) Calculation of gradient vectors 

For each pixel j among all of the pixels constituting 
a given image, the orientation 0 of the gradient vector of the 
image signal representing the image is calculated with 
Formula (1) shown below. 

0 - tan" 1 (/j + h + /s + h + fl ) " (/ " + 7,2 + /l3 +f » +f » ) (i) 
(/, + /, + /a + /. s + /. 6 ) " (fi + ft + A + fx o + /., ) 

As illustrated in Figure 3, f 1 through f 16 in Formula 

(1) represent the pixel values (i.e. , the image signal values) 
corresponding to the pixels located at the peripheral areas 
of a mask, which has a size of , for example, five pixels (located 
along the column direction of the pixel array) x five pixels 
(located along the row direction of the pixel array) and which 
has its center at the pixel j . 

(Step 2) Calculation of the degree of convergence of 
gradient vectors 

Thereafter, for each pixel among all of the pixels 
constituting the given image is taken as a pixel of interest, 
and the degree of convergence C of the gradient vectors with 
respect to the pixel of interest is calculated with Formula 

(2) shown below. 

n 

C = (l/ AO£cos0 y (2) 

j=\ 
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As illustrated in Figure 4, in Formula (2), 
N represents the number of pixels located in the region inside 
of a circle, which has its center at the pixel of interest and 
has a radius R, and 0., represents the angle between the straight 
line, which connects the pixel of interest and each pixel j 
located in the circle, and the gradient vector at the pixel j , 
which has been calculated with Formula (1) . Therefore, incases 
where the orientations of the gradient vectors of the respective 
pixels j converge toward the pixel of interest, the degree of 
convergence C represented by Formula {2) takes a large value. 

The gradient vector of each pixel j , which is located 
in the vicinity of a tumor pattern, is directed approximately 
to the center portion of the tumor pattern regardless of the 
level of contrast of the tumor pattern. Therefore, it can be 
regarded that the pixel of interest associated with the degree 
of convergence C, which takes a large value, is the pixel located 
at the center portion of the tumor pattern. On the other hand, 
in a linear pattern, such as a blood vessel pattern, the 
orientations of the gradient vectors are biased to a certain 
orientation, and therefore the value of the degree of 
convergence C is small. Accordingly, a tumor pattern can be 
detected by taking each of all pixels, which constitute the 
image, as the pixel of interest, calculating the value of the 
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degree of convergence C with respect to the pixel of interest, 
and rating whether the value of the degree of convergence C 
is larger than a predetermined threshold value . Specifically, 
the processing with the iris filter has the features over an 
ordinary difference filter that the processing with the iris 
filter is not apt to be adversely affected by blood vessel 
patterns, mammary gland patterns, or the like, and can 
efficiently detect tumor patterns. 

In actual processing, such that the detection 
performance is unaffected by the sizes and shapes of tumor 
patterns may be achieved, it is contrived to adaptively change 
the size and the shape of the filter. Figure 5 shows an example 
of the filter. The filter is different from the filter shown 
in Figure 4. With the filter of Figure 5, the degree of 
convergence is rated only with the pixels, which are located 
along radial lines extending radially from a pixel of interest 
in M directions adjacent at 27i/M degree intervals . (In Figure 5 
by way of example, 32 directions at 11.25 degree intervals are 
shown. ) 

In cases where the pixel of interest has the 
coordinates (k, 1), the coordinates ( [x] , [y] ) of the pixel, 
which is located along an i th radial line and is the n th pixel 
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as counted from the pixel of interest , are given by Formulas ( 3 ) 
and (4) shown below. 

x=k+n cos {27t(i-l)/M} (3) 
y=l+n sin {2jt(i-l)/M} (4) 
wherein [x] represents the maximum integer, which does not 
exceed x, and [y] represents the maximum integer, which does 
not exceed y. 

Also, for each of the radial lines, the output value 
obtained for the pixels ranging from a certain pixel to a pixel , 
which is located along the radial line and at which the maximum 
degree of convergence is obtained, is taken as the degree of 
convergence Ci max with respect to the direction of the radial 
line. The mean value of the degrees of convergence Ci max , which 
have been obtained for all of the radial lines, is then calculated 
The mean value of the degrees of convergence Ci max having thus 
been calculated is taken as the degree of convergence C of the 
gradient vector group with respect to the pixel of interest. 

Specifically, the degree of convergence Ci (n) , which 
is obtained for the pixels ranging from the pixel of interest 
to the n ch pixel located along the i th radial line, is calculated 
with Formula (5) shown below. 

C/(») = 2 {(cos Q a ) / n) , Rmm<n<R max (5) 



wherein Rmin and Rmax respectively represent the minimum value 
and the maximum value having been set for the radius of the 
tumor pattern, which is to be detected. 

Specifically, with Formula (5) , the degree of 
convergence Ci (n) is calculated with respect to all of the pixels , 
which are located along each of the radial lines and fall within 
the range from a starting point to an end point . The starting 
point is set at the pixel of interest. The end point is set 
at one of the pixels that are located between a position at 
the length of distance corresponding to the minimum value Rmin 
having been set for the radius of the tumor pattern, which is 
to be detected, and a position at the length of distance 
corresponding to the maximum value Rmax. 

Thereafter, the degree of convergence C of the 
gradient vector group is calculated with Formulas (6) and (7) 
shown below. 

C/ = max Ci(n) (6) 

max «minS«S«max 
32 

C = (l/32)]TC7 max (7) 
<=i 

The value of Ci max of Formula (6) represents the maximum 
value of the degree of convergence Ci (n) obtained for each of 
the radial lines with Formula (5) . Therefore, the region from 



the pixel of interest to the pixel associated with the degree 
of convergence Ci (n) , which takes the maximum value, may be 
considered as being the region of the tumor pattern candidate 
along the direction of the radial line. 

The calculation with Formula (6) is made for all of 
the radial lines, and the contours (marginal points) of the 
regions of the tumor pattern candidate on all of the radial 
lines are thereby detected. The marginal points of the regions 
of the tumor pattern candidate on the adjacent radial lines 
are then connected by a straight line or a non- linear curve. 
In this manner, it is possible to specify the shape of the outer 
periphery of the region, which may be regarded as the tumor 
pattern candidate . 

Thereafter, with Formula (7) , the mean value of the 
maximum values Ci max of the degrees of convergence within the 
aforesaid regions, which have been obtained by Formula (6) for 
all directions of the radial lines, is calculated. In 
Formula (7) , by way of example, the radial lines are set along 
32 directions. The calculated mean value serves as an output 
value I of the iris filtering processing. The output value I 
is compared with a predetermined constant threshold value Tl, 
which is appropriate for making a judgment as to whether the 
detected pattern is or is not a tumor pattern candidate. In 



cases where I>T1 (or I>T1) , it is judged that the region having 
its center at the pixel of interest is an abnormal pattern 
candidate {a tumor pattern candidate) . In cases where I<T1 
(or I<T1) , it is judged that the region having its center at 
the pixel of interest is not a tumor pattern candidate. 

The calculation of the degree of convergence Ci (n) 
may be carried out by using Formula (5') shown below in lieu 
of Formula (5) . 

Ci(n) = Y cos 0„ , R min <n< R max (5' ) 

w-tfmin+l/^in 

Specifically, with Formula (5 1 ), the degree of 
convergence Ci (n) is calculated with respect to all of the pixels, 
which are located along each of the radial lines and fall within 
the range from a starting point to an end point. The starting 
point is set at a pixel that is located at the length of distance 
corresponding to the minimum value Rmin having been set for 
the radius of the tumor pattern to be detected, which is taken 
from the pixel of interest. The end point is set at one of 
the pixels that are located between the position at the length 
of distance corresponding to the minimum value Rmin and the 
position at the length of distance corresponding to the maximum 
value Rmax, which is taken from the pixel of interest. 
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The morphological operation processing is a 
technique for detecting a candidate for a microcalcif ication 
pattern, which is one of the characteristic forms of mammary 
cancers as in the cases of the tumor patterns . The morphological 
operation processing is performed by using a multi-scale X and 
a structure element (i.e., a mask) B. The morphological 
operation processing has the features that , for example, (1) it 
is efficient for extracting a calcification pattern itself, 
(2) it is not affected by complicated background information, 
and (3) the extracted calcification pattern does not become 
distorted. Specifically, the morphological operation 
processing is advantageous over ordinary differentiation 
processing in that it can more accurately detect geometrical 
information concerning the size, the shape, and the density 
distribution of the calcification pattern. The morphological 
operationprocessing is performed in the manner described below. 
(Fundamental morphological operation) 

In general, the morphological operation processing 
is expanded as the theory of sets in an N-dimensional space. 
As an aid in facilitating intuitive understanding, the 
morphological operation processing will be described 
hereinbelow with reference to a two-dimensional gray level 
image . 
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The gray level image is considered as a space, in 
which a point having coordinates (x, y) has a height 
corresponding to an image density value f (x, y) . In this case, 
it is assumed that the image signal representing the image 
density value f (x, y) is a high luminance -high signal level 
type of image signal, in which low image density (i.e. , a high 
luminance when the image is displayed on a CRT display device) 
is represented by high image signal levels. 

Firstly, as an aid in facilitating the explanation, 
a one -dimensional function f (x) corresponding to the 
cross -section of the two-dimensional gray level image is 
considered. It is assumed that a structure element g used in 
the morphological operation processing is a symmetric function 
of Formula (8) , which is symmetric with respect to the origin. 

g s (x)=g(-x) (8) 
It is also assumed that the value is 0 in a domain of definition G, 
which is represented by Formula (9) . 

G={-m, -m+1, -1, 0, 1, m-1, m} (9) 

In such cases, the fundamental forms of the 
morphological operation are very simple operations performed 
with Formulas (10), (11), (12), and (13) shown below. 



33 



Dilation: 



[f0G s ] (i) =max{f (i-m) , 



f (i) , 



f (i+m) } 



(10) 



Erosion: 



[f3^G s ] (i) =min{f (i-m) , 



f(i), f(i+m)} 



(11) 



Opening: f g =(f^g s )©g 



(12) 



Closing: f g =(f0g s )^g 



(13) 



Specifically, 



as illustrated in Figure 6A, the 



dilationprocessing is the processing for retrieving the maximum 
value in a width of ±m (the value determined in accordance with 
a structure element B) having its center at a pixel of interest. 
As illustrated in Figure 6B, the erosion processing is the 
processing for retrieving the minimum value in the width of 
±m having its center at the pixel of interest. The opening 
processing is equivalent to searching for the maximum value 
after searching for the minimum value. Also, the closing 
processing is equivalent to searching for the minimum value 
after searching for the maximum value . More specifically, as 
illustrated in Figure 6C, the opening processing is equivalent 
to a process for smoothing the image density curve f (x) from 
the low luminance side, and removing a convex image density 
fluctuating area (i.e. , the area at which the luminance is higher 
than that of the surrounding areas) , which fluctuates in a range 
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spatially narrower than the mask size of 2m. Also, as 
illustrated in Figure 6D, the closing processing is equivalent 
to a process for smoothing the image density curve f (x) from 
the high luminance side, and removing a concave image density 
fluctuating area (i.e. , the area at which the luminance is lower 
than that of the surrounding areas) , which fluctuates in a range 
spatially narrower than the mask size of 2m. 

In cases where the image signal representing the image 
density value f (x) is a high image density-high signal level 
type of image signal , in which a high image density is represented 
by a high image signal level, the relationship between the image 
density value f (x) and the image signal value becomes reverse 
to the relationship between the image density value f (x) and 
the image signal value in the high luminance-high image signal 
level type of image signal. Therefore, the dilation processing 
which is performed on the high image density-high signal level 
type of image signal, corresponds to the erosion processing, 
which is performed on the high luminance -high signal level type 
of image signal as shown in Figure 6B . The erosion processing, 
which is performed on the high image density-high signal level 
type of image signal, corresponds to the dilation processing, 
which is performed on the high luminance -high signal level type 
of image signal as shown in Figure 6A. The opening processing, 
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which is performed on the high image density-high signal level 
type of image signal, corresponds to the closing processing, 
which is performed on the high luminance -high signal level type 
of image signal as shown in Figure 6D. Also, the closing 
processing, which is performed on the high image density-high 
signal level type of image signal, corresponds to the opening 
processing, which is performed on the high luminance-high signal 
level type of image signal as shown in Figure 6C. 

The morphological operation processing will be 
described with respect to the high luminance -high signal level 
type of image signal (i.e., the image signal representing the 
luminance value) . 

(Application to detection of calcification patterns) 

In order for a calcification pattern to be detected, 
a difference method, in which a smoothed image signal is 
subtracted from the original image signal, may be considered. 
However, with a simple smoothing method, it is difficult to 
discriminate the calcification pattern from an elongated 
non- calcification pattern (for example, a pattern of the mammary 
gland, a blood vessel, mammary gland supporting tissues, or 
the like) . Therefore, Kobatake, et al . have proposed 
morphological operation processing, which is represented by 
Formula ( 14 ) shown below and is based upon the opening operation 
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using a multi- structure element. [Reference should be made 

to "Extraction of Microcalcif ications Using Morphological 

Filter with Multiple Structuring Elements," Transactions of 

The Institute of Electronics, Information, and Communication 

Engineers of Japan, D-II, Vol. J75-D-II, No. 7, pp. 1170-1176, 

July 1992; and "Fundamentals of Morphology and Its Application 

to Mammogram Processing, " Medical Imaging Technology, Vol. 12, 

No. 1, January 1994.] 

P = f-max M) {(/^Bi)®Bi} 

= f- max {f Bi } (14) 

;e(l M) 

In Formula (14) , Bi (wherein i = l, 2, . . . , M) represents, 
for example, four linear structure elements B (in this case, 
M=4) shown in Figure 7. (The four structure elements, as a 
whole, will hereinbelow be referred to as the multi -structure 
element.) In cases where the structure element B is set to 
be larger than the calcification pattern to be detected, a 
calcification pattern, which is a convex signal change area 
finer than the structure element B (i.e., which is an image 
area fluctuating in a spatially narrow range) , is removed in 
the opening processing. On the other hand, an elongated 
non-calcification pattern is longer than the structure 
element B. Therefore, in cases where the inclination of the 



37 



non-calcification pattern {i.e., the direction along which the 
non-calcification pattern extends) coincides with one of the 
directions of the four structure elements Bi, the 
non-calcification pattern remains unremoved after the opening 
processing, i.e. the operation of the second term of Formula (14) , 
has been performed. Therefore, in cases where the smoothed 
image signal obtained from the opening processing (i.e., the 
signal representing the image, from which the calcification 
pattern has been removed) is subtracted from the original image 
signal f , an image is capable of being obtained which contains 
only the microcalcif ication pattern candidate. This is the 
concept behind Formula (14) . 

As described above, in cases where the image signal 
is of the high image density-high signal level type, the image 
density value of the calcification pattern is smaller than the 
image density values of the surrounding image areas, and the 
calcification pattern constitutes a concave signal change area 
with respect to the surrounding areas. Therefore, the closing 
processing is applied in lieu of the opening processing, and 
Formula (15) shown below is applied in lieu of Formula (14) . 
P = f- min {{f®Bi)^.Bi} 

J /€(! AO 

''-A"** 1151 
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However, it often occurs that a non-calcification 
pattern having the same size as the size of the calcification 
pattern remains in the obtained image . In such cases, the signal, 
which represents the non-calcification pattern and is contained 
in P of Formula (14) , is removed by utilizing the differentiation 
information based upon the morphological operation performed 
with Formula (16) shown below. 

M grad = (l/2)x{f0XB-f^XB} (16) 

A large value of M grad indicates a high possibility 
of being a calcification pattern. Therefore, a calcification 
pattern candidate Cs is capable of being detectedby Formula ( 17 ) 
shown below. 

If P(i, j)>Tl and M grad (i, j ) >T2 

then Cs(i, j ) =P else Cs(i, j ) =0 (17) 

In Formula (17) , TlandT2 represent the predetermined 
threshold values, which are capable of being determined 
experimentally . 

However, a non-calcification pattern, which has a 
size different from the size of the calcification pattern, can 
be removed by only the comparison of P of Formula (14) and the 
predetermined threshold value Tl . Therefore, in cases where 
there is no risk that a non-calcification pattern having the 
same size as the size of the calcification pattern remains, 
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it is sufficient for the condition of the first term of 
Formula (17) , i.e. , the condition of P ( i , j)>Tl, to be satisfied . 

Finally, the cluster Cc of the calcification pattern 
is detected by the combination of the opening operation and 
the closing operation of the multi-scale in accordance with 
Formula (18) shown below. 

Cc=Cs©X 1 B^^ 3 B©X 2 B (18) 

In Formula (18) , X 1 and X 2 are respectively determined 
by the maximum distance of the calcification pattern to be 
combined and the maximum radius of the isolated pattern to be 
removed, and X 3 =X 1 +X 2 . 

As for the high luminance -high signal level type of 
image signal, the morphological operation processing is 
conducted in the manner described above . In cases where the 
image signal is of the high image density-high signal level 
type ( in which a pixel of a high image density has a large digital 
signal value) , the relationship between the opening operation 
and the closing operation is reversed. 

(Processing for calculating the likelihood ratio of the 
Mahalanobis distance) 

The Mahalanobis distance is one of distance scales, 
which are utilized for image pattern recognition. Similarity 
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of an image pattern is capable of being found from the value 
of the Mahalanobis distance. With the processing utilizing 
the Mahalanobis distance, a plurality of feature measurements 
representing the features of an image pattern are expressed 
by vectors, and the Mahalanobis distance is defined such that 
the differences in vectors between a reference image and an 
image subjected to the recognition may be reflected. Examples 
of the feature measurements utilized for the calculation of 
the Mahalanobis distance include those described below. 

A first feature measurement is circularity Sp of a 
region (hereinbelow referred to the calculation obj ect region) , 
which is subj ected to the calculation of the Mahalanobis distance 
The circularity Sp represents the feature of the contour shape 
of the calculation object region. As illustrated in Figure 8, 
an area A of the calculation obj ect region and a center of gravity 
AO on the calculation obj ect region are calculated, and a virtual 
circle is set. The virtual circle has an area approximately 
equal to the area A of the calculation object region, has its 
center at the position at which the center of gravity AO is 
located, and has a radius R. Further, an occupation ratio of 
the calculation object region, which is contained within the 
virtual circle, with respect to the area A is calculated as 
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the circularity Sp . Specifically, the circularity Sp is 
calculated with Formula (19) shown below. 

Sp=A'/nR 2 (19) 
wherein A' represents the area of the overlapping region, at 
which the virtual circle and the calculation object region 
overlaps one upon the other. 

Also, as the feature measurements of the interior 
of the calculation obj ect region, the three feature measurements 
described below are calculated. Specifically, a histogram of 
image density values S of the calculation obj ect region is formed 
The frequency of occurrence of each of the image density values S 
is represented by P(S) . In accordance with the distribution 
of the frequencies of occurrence P(S) with respect to the image 
density values S , a second feature measurement , which represents 
the variance, var, is calculated with Formula (20) . Also, a 
third feature measurement , which represents the contrast, con, 
is calculated with Formula (21) . Further, a fourth feature 
measurement, which represents the angular moment, asm, is 
calculated with Formula (22) . 

var = S) 2 ■ P(S)} (20) 

con = jT{S 2 -P(S)} (21) 
asm = Y j {P(S)} 2 (22) 
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wherein S represents the mean value of the image density values 
S within the region, and N represents the number of pixels falling 
within the region. 

Further, as the feature measurements of the periphery 
of the calculation object region, five feature measurements 
described below are calculated in accordance with an iris filter 
edge image { I FED image) . How the five feature measurements 
are calculated will be described hereinbelow. 

With respect to the candidate region, i.e., the tumor 
pattern Pj representing a breast cancer in the radiation image 
or a false abnormal pattern P M , which has been detected by the 
iris f ilteringprocessing, an image area containing the detected 
region and the neighboring region is detected as, for example, 
a square region. Also, as for the thus detected square region, 
a peripheral edge image (the IFED image) is formed by the 
utilization of the iris filtering processing. 

Specifically, with the iris filtering processing, 
the position of the point, which gives the maximum value of 
the degree of convergence Ci (n) on the i th radial line extending 
radially from the pixel of interest, the maximum value being 
calculated with Formula (6) , is detected. In this processing, 



no limitation is imposed upon the value of n giving the maximum 
value of the degree of convergence Ci (n) . 

As a result, in cases where the pixel of interest 
is located within the candidate region Pj or P M/ the value of 
n giving the maximum value of Formula {6) indicates the pixel, 
at which the i ch radial line intersects with the periphery B 
of the candidate region Pj or P M . For example, as illustrated 
in Figure 9 , as for a pixel of interest 1 , the value of n indicates 
pixels B 1# B 2 , B 3 , and B 4 . As for a pixel of interest 2 shown 
in Figure 9, the value of n indicates pixels B 2 , B 5 , B 6 , and B 7 . 

In cases where the pixel of interest is located in 
the region outward from the candidate region Pj or P M , when the 
value of n indicates the pixel of interest itself, the value 
of Formula (6) takes the maximum value. Specifically, as for 
a pixel of interest 3 shown in Figure 9, which is located in 
the region outward from the candidate region Pj or P M , the value 
of Formula (6) takes the maximum value when the value of n 
indicates the pixel of interest itself . 

All of the pixels falling within the square region, 
which contains the candidate region, are successively taken 
as the pixel of interest, and the number of pixels, which are 
associated with the maximum value of Formula (6) , is counted. 
As a result, an image shown in Figure 10 is obtained. 
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Specifically, as for all of the pixels, which are 
located in the region outside the candidate region Pj or P M , 
a count value of "1" is obtained. As for all of the pixels, 
which are located within the candidate region Pj or P M , a count 
value of "0" is obtained. Also, as for all of the pixels, which 
are located on the periphery B of the candidate region P a or 
P M , count values larger than 1 are obtained. The image 
representing the count values is defined as the I FED image. 
In this manner, the I FED image is formed. 

Thereafter, the processing described below is 
performed on the I FED image, and a co-occurrence matrix is 
formed. 

Specifically, as illustrated in Figure 11, the 
position, at which the center of gravity AO on the candidate 
region P a or P M is located, is found. A radial line is extended 
from the position, at which the center of gravity AO is located. 
An arbitrary point lying on the radial line is represented by i , 
and a point, which is spaced a distance equal to the sum of 
the lengths of two pixels from the point i along a line normal 
to the radial line, is represented by j . 

The count value at the point i on the I FED image and 
the count value at the point j on the I FED image are counted 
up on a matrix shown in Figure 12 . Specifically, in cases where 
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the point i is located in the region outward from the candidate 
region Pj or P M , the count value at the point i on the I FED image 
is "1 . " At this time, in cases where the point j is also located 
in the region outward from the candidate region P a or P M , the 
count value at the point j on the IFED image is "1." In such 
cases, on the matrix shown in Figure 12, "1" is counted in the 
cell, which is located at the intersection of an i-row "1" and 
a j -column "1 . " 

In cases where the point i is located within the 
candidate region Pj or P M , and the point j is also located within 
the candidate region P a or P M , the count values at the points i 
and j on the IFED image are "0." In such cases, on the matrix 
shown in Figure 12, "1" is counted in the cell, which is located 
at the intersection of an i-row "0" and a j -column "0." 

Also, in cases where the point i is located at the 
periphery B of the candidate region Pj or P M , and the point j 
is also located at the periphery B of the candidate region Pj 
or P M , the count value at the point i on the IFED image is, 
for example, "5," and the count value at the point j on the 
IFED image is, for example, "3." In such cases, on the matrix 
shown in Figure 12, "1" is counted in the cell, which is located 
at the intersection of an i-row "5" and a j -column "3." The 
count value, which is counted up on the matrix, is cumulated. 
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Specifically, when the point i on the IFED image, at which the 
count value is "5," and the corresponding point j on the IFED 
image, at which the count value is "3," are found again, "1" 
is added to the previously counted value "1" in the cell, which 
is located at the intersection of the i-row "5" and the j -column 
"3," and therefore "2" is stored in the cell. 

The point i is an arbitrary point on the IFED image. 
The radial lines are set such that all of the pixels of the 
IFED image may be taken as the point i , and the point i is searched 
for along the respective radial lines. In this manner, the 
matrix is completed. The matrix of the IFED image is referred 
to as the co-occurrence matrix P g (x, y) . 

The shape of a tumor pattern has characteristics such 
that the periphery of the tumor pattern has an approximately 
circular shape. Also, the points i and j are very close to 
each other. Therefore, in cases where the candidate region 
is the tumor pattern, there is a high level of probability that, 
when the point i is located at the periphery of the tumor pattern 
(i.e., when the count value at the point i on the IFED image 
is larger than 1) , the point j will also be locatedat the periphery 
of the tumor pattern (i.e., the count value at the point j on 
the IFED image will be larger than 1) . 
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On the other hand, in cases where the candidate region 
is a false abnormal pattern, as in the pattern at which two 
blood vessel patterns intersect each other, it is very rare 
that the false abnormal pattern will have a circular periphery. 
Therefore, even if the points i and j are close to each other, 
when the point i is located at the periphery of the false abnormal 
pattern, the point j will not necessarily be located at the 
periphery of the false abnormal pattern. In such cases, the 
probability that the point j will also be located at the periphery 
of the false abnormal pattern is markedly low. 

Therefore, in accordance with whether the candidate 
region is the tumor pattern or the false abnormal pattern, 
significant differences are found in the characteristic values 
of the co-occurrence matrix P g (x, y) . The characteristic values 
of the co-occurrence matrix are herein referred to as edge 
information. The edge information is utilized as the feature 
measurement. Specifically, as shown below, a fifth feature 
measurement, which represents the variance, var, with respect 
to the co-occurrence matrix is calculated with Formula (23) . 
Also, a sixth feature measurement, which represents the 
difference entropy, dfe, with respect to the co-occurrence 
matrix is calculated with Formula (24) . Further, a seventh 
feature measurement, which represents the correlation, cor, 
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with respect to the co-occurrence matrix is calculated with 
Formula (25) . Furthermore, an eighth feature measurement, 
which represents the inverse difference moment, idm, with 
respect to the co-occurrence matrix is calculated with 
Formula (26) . Also, a ninth feature measurement, which 
represents the sum entropy , se, with respect to the co-occurrence 
matrix is calculated with Formula (27) . 

var = XZW-/0 2 P g {iJ)} (23) 
< j 

dfe = £{/^(£)-log|/>,_,(*)|} (24) 

k 

cor = Z S [ 0 • J ■ p g ('"' J) ~ Mx ' My) / " <r, )] (25) 
idm = YL\- P S C y) / 0 + " -/") 2 >] (26) 
« = -S[ /, «>(*)" 1 °8< /, «/*»] (27) 

wherein 

P X+ yW = ZZ P A i 'J^ k=i + J 

i J 

P x (i) represents the projection distribution along the j 
direction, i.e., 

y 

and P y (j) represents the projection distribution along the i 
direction, i.e., 



49 



In accordance with the thus obtained feature 
measurements of the calculation object region, a Mahalanobis 
distance Dml from a pattern class (i=l) , which represents a 
non-malignant pattern, and a Mahalanobis distance Dm2 from a 
pattern class (i=2) , which represents a malignant pattern, are 
calculated with Formula (28) shown below. 

Dmi = (x-mi)'^~\x-mi) (28) 

wherein Si represents the covariance matrix of the pattern class 
(pattern classification between the non-malignant pattern of 
i=l and the malignant pattern of i=2) wi, i.e., 
=(1/M) £(*-mO(x -mi)' 

t represents the transposed vector (row vector) , x represents 
the vector of the feature measurement x, i.e., 

x = (xl , x2 , . . . , xN) 
Li" 1 represents the inverse matrix of li, and mi represents the 
mean value of the pattern class wi, i.e., 
mi = (1/M)£ic 

The obtained feature measurements correspond to xl 
to xN described above and express an N-dimensional space with 
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the form of (xl, x2 , x3 , . . . , xN) . The Mahalanobis distance 
between the pattern class of the calculation object region 
expressed on the N-dimensional pattern space and the pattern 
class of the non-malignant pattern is represented by Dml . Also , 
the Mahalanobis distance between the pattern class of the 
calculation object region expressed on the N-dimensional 
pattern space and the pattern class of the malignant pattern 
is represented by Dm2 . 

Each of the pattern class of the non-malignant pattern 
and the pattern class of the malignant pattern represents the 
class of the pattern space, which has been set in accordance 
with the results of experiments carried out on a plurality of 
abnormal pattern candidates. Each pattern class is defined 
by the vector x with respect to each of the non-malignant 
pattern and the malignant pattern. For example, the pattern 
class of the non-malignant pattern is represented by the pattern 
class wl, which is formed with the mean value of the vector 
x with respect to the pattern regarded as being the 
non-malignant pattern. Also, the pattern class of the 
malignant pattern is represented by the pattern class w2 , which 
is formed with the mean value of the vector x with respect to 
the pattern regarded as being the malignant pattern. 
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In cases where the calculation object region is the 
malignant pattern, the Mahalanobis distance with respect to 
the pattern class of the malignant pattern is apt to be short 
{i.e. , Dm2 is apt to take a small value) , and the Mahalanobis 
distance with respect to the pattern class of the non-malignant 
pattern is apt to exhibit a variation. Also, in cases where 
the calculation object region is the non-malignant pattern, 
the Mahalanobis distance with respect to the pattern class of 
the non-malignant pattern is apt to be short (i.e. , Dml is apt 
to take a small value) , and the Mahalanobis distance with respect 
to the pattern class of the malignant pattern is apt to exhibit 
a variation. Therefore, the likelihood ratio of the 
Mahalanobis distance, which is effective for discriminating 
the malignant pattern and the non-malignant pattern from each 
other in accordance with the tendency described above, is 
calculated for each of calculation object regions. 

The likelihood ratio of the Mahalanobis distance is 
represented by Dml/Dm2 . The likelihood ratio represents the 
slope on a coordinate plane of Figure 13. Specifically, in 
cases where the likelihood ratio is high, it may be judged that 
there is a high possibility of the pattern being the malignant 
pattern. Also, in cases where the likelihood ratio is low, 
it may be judged that there is a high possibility of the pattern 
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being the non-malignant pattern. Therefore, for example, the 
threshold value may be set at 2. Also, a judgment may be made 
such that a pattern associated with the likelihood ratio of 
at least 2 is to be detected as the abnormal pattern candidate, 
and a pattern associated with the likelihood ratio lower than 
2 is not to be detected as the abnormal pattern candidate. 

The iris filtering processing, the morphological 
filtering processing, and the processing for calculating the 
likelihood ratio of the Mahalanobis distance are performed in 
the manner described above . 

With the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the calculation is perf ormedperf ormed to find the 
degree of certainty about malignancy , which represents the level 
of possibility of a pattern being a malignant pattern, with 
respect to the abnormal pattern candidate detected by the 
abnormal pattern candidate detection processing. The 
calculation is perf ormedperf ormed in accordance with the index 
value representing the feature of the abnormal pattern candidate 
and in accordance with the correlation between the index value 
and possibility of a pattern being a malignant pattern, which 
has been obtained from clinical results . Also, the information 
representing the degree of certainty about malignancy with 
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respect to the abnormal pattern candidate is outputted and 
displayed together with the medical image and the information 
for specifying the detected abnormal pattern candidate. 
Therefore, with the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the specific information useful for diagnosis is 
capable of being furnished to the operator who views the 
displayed medical image. Accordingly, the capability of the 
outputted image serving as an effective tool in, particularly, 
the efficient and accurate diagnosis of an illness, is capable 
of being enhanced even further. 

With the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, wherein the index value is the index value utilized 
for the detection of the abnormal pattern candidate, higher 
significance with respect to the correlation between the index 
value and the possibility of a pattern being a malignant pattern 
is capable of being expected than the cases where an index value 
other than the index value utilized for the detection of the 
abnormal pattern candidate is employed. Therefore, the 
reliability of the calculated degree of certainty about 
malignancy is capable of being enhanced. 
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The first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention may be modified such that the information for 
specifying the detected abnormal pattern candidate and the 
information representing the degree of certainty about 
malignancy with respect to the abnormal pattern candidate is 
a mark. The mark is displayed at the position to indicate the 
abnormal pattern candidate on the medical image, such that the 
kind of the mark may be altered in accordance with the degree 
of certainty about malignancy . With the modification described 
above , both the information for specifying the detected abnormal 
pattern candidate and the information representing the degree 
of certainty about malignancy with respect to the abnormal 
pattern candidate are capable of being displayed by the single 
mark. Therefore, the information for specifying the detected 
abnormal pattern candidate and the information representing 
the degree of certainty about malignancy with respect to the 
abnormal pattern candidate are capable of being recognized 
easily. 

With the second abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the medical image and the information for specifying 
the detected abnormal pattern candidate are outputted and 
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displayed, and the information representing the degree of 
certainty about malignancy with respect to an arbitrary image 
region selected in the medical image is displayed in the vicinity 
of the selected image region. Therefore, besides the detected 
abnormal pattern candidate, the information representing the 
degree of certainty about malignancy with respect to the 
arbitrary image region is capable of being displayed andutilized 
for comparison. Accordingly, the capability of the outputted 
image serving as an effective tool in, particularly, the 
efficient and accurate diagnosis of an illness, is capable of 
being enhanced even further. 

With the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention and the second abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, wherein the information representing the degree of 
certainty about malignancy is the numerical value, recognition 
of fine differences becomes possible. Therefore, comparison 
between a detected abnormal pattern candidate and other 
candidates or a selected image region is capable of being made 
easily. 

With the first abnormal pattern candidate detection 
processing method and system in accordance with the present 
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invention and the second abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the information representing the degree of certainty 
about malignancy may be the warning message. In this case, 
the warning message is altered in accordance with the degree 
of certainty about malignancy. Accordingly, the directly 
understandable information is capable of being given to the 
operator who views the displayed medical image. Therefore, 
the state of the abnormal pattern candidate is capable of being 
understood easily. 

With the third abnormal pattern candidate detection 
processing method and system in accordance with the present 
invention, the medical image and the information for specifying 
the detected abnormal pattern candidate are outputted and 
displayed, and the distribution image corresponding to the 
medical image, which represents the distribution of the degrees 
of certainty about malignancy, is displayed. Therefore, the 
state of distribution of the degrees of certainty about 
malignancy with respect to the entire pattern of the medical 
image is capable of being found. Accordingly, the width of 
the approach to diagnosis becomes broad, and the capability 
of the outputted image serving as an effective tool in, 
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particularly, the efficient andaccurate diagnosis of an illness , 
is capable of being enhanced even further. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram showing a first embodiment 
of the abnormal pattern candidate detection processing system 
in accordance with the present invention, 

Figure 2A is an explanatory view showing an example 
of a mammogram, 

Figure 2B is an explanatory view showing the degree 
of convergence of gradient vectors in a tumor pattern, 

Figure 2C is an explanatory view showing the degree 
of convergence of gradient vectors in an elongated pattern, 
such as a blood vessel pattern or a mammary gland pattern, 

Figure 2D is an explanatory view showing the degree 
of convergence of gradient vectors in an area at which two 
elongated patterns , such as mammary gland patterns , intersect 
each other, 

Figure 3 is an explanatory view showing a mask, which 
has its center at a pixel of interest j and has a size of 5 
pixels (along a column direction) x 5 pixels (along a row 
direction) , 

Figure 4 is an explanatory view showing an angle made 
between a pixel of interest and a gradient vector at a pixel j , 
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Figure 5 is an explanatory view showing the concept 
behind an iris filter, which is set such that a contour shape 
may change adaptively, 

Figure 6A is a graph showing how dilation processing, 
which is one of fundamental operations with a morphological 
filter, is performed, 

Figure 6B is a graph showing how erosion processing, 
which is one of fundamental operations with a morphological 
filter, is performed, 

Figure 6C is a graph showing how opening processing, 
which is one of fundamental operations with a morphological 
filter, is performed, 

Figure 6D is a graph showing how closing processing, 
which is one of fundamental operations with a morphological 
filter, is performed, 

Figure 7 is an explanatory view showing four linear 
structure elements employed in a morphological filter, 

Figure 8 is an explanatory view showing a virtual 
circle having an area equivalent to an area A of a candidate 
region, 

Figure 9 is an explanatory view showing how an iris 
filter edge (IFED) image is formed, 

Figure 10 is an explanatory view showing an IFED image, 
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Figure 11 is an explanatory view showing how a 
co-occurrence matrix is formed in accordance with the I FED image, 

Figure 12 is an explanatory view showing a 
co-occurrence matrix, 

Figure 13 is an explanatory graph showing how a 
j udgment is performed in accordance with a Mahalanobis distance , 

Figure 14 is a graph showing a relationship between 
a likelihood ratio Y of a Mahalanobis distance and a proportion 
occupied by each of a malignant state, a benign state, and a 
normal state in an extracted candidate pattern, which 
relationship has been obtained from clinical results, 

Figure 15 is a schematic view showing information 
outputted to a CRT display device in the first embodiment of 
the abnormal pattern candidate detection processing system in 
accordance with the present invention, 

Figure 16 is a block diagram showing a second 
embodiment of the abnormal pattern candidate detection 
processing system in accordance with the present invention, 

Figure 17 is a schematic view showing information 
outputted to a CRT display device in the second embodiment of 
the abnormal pattern candidate detection processing system in 
accordance with the present invention, 
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Figure 18 is a block diagram showing a third embodiment 
of the abnormal pattern candidate detection processing system 
in accordance with the present invention, 

Figure 19 is a schematic view showing information 
outputted to a CRT display device in the third embodiment of 
the abnormal pattern candidate detection processing system in 
accordance with the present invention, 

Figure 20 is a graph showing relationship between 
a threshold value of the likelihood ratio of the Mahalanobis 
distance at the time of detection of an abnormal pattern 
candidate and a malignant pattern detection rate, which 
relationship has been obtained from clinical results, and 

Figure 21 is a schematic view showing an example of 
outputting to a CRT display device in the abnormal pattern 
candidate detection processing system. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention will be described in further 
detail with reference to the accompanying drawings. 

Firstly, a first embodiment, which is an embodiment 
of the first abnormal pattern candidate detection processing 
system in accordance with the present invention, will be 
described hereinbelow. 
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Figure 1 is a block diagram showing a first embodiment 
of the abnormal pattern candidate detection processing system 
in accordance with the present invention. With reference to 
Figure 1, an abnormal pattern candidate detection processing 
system 100 receives an image signal representing a mammogram 
(i.e., a medical image of the breast) P of a patient and outputs 
an image signal representing the mammogram and other kinds of 
information to external image displaying means (such as a CRT 
display device) . The abnormal pattern candidate detection 
processing system 100 comprises abnormal pattern candidate 
detecting means 10 for performing abnormal pattern candidate 
detection processing on the received image signal P, thereby 
detecting an abnormal pattern candidate PI embedded in the image 
P, which is represented by the image signal P. (As an aid in 
facilitating the explanation, an image and an image signal 
representing the image are herein represented by the same 
reference letter.) The abnormal pattern candidate detection 
processing system 100 also comprises malignancy certainty 
degree calculatingmeans 20 for calculating a degree of certainty 
about malignancy D, which represents a level of possibility 
of a pattern being a malignant pattern, with respect to the 
abnormal pattern candidate PI . The calculation of the degree 
of certainty about malignancy D is performed in accordance with 
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an index value representing a feature of the abnormal pattern 
candidate PI and in accordance with a correlation between the 
index value and the possibility of a pattern being a malignant 
pattern, which has been obtained from clinical results. The 
abnormal pattern candidate detection processing system 100 
further comprises image output means 30. As information for 
specifying the detected abnormal pattern candidate PI , the image 
output means 3 0 displays an arrow mark at a position to indicate 
the abnormal pattern candidate PI in the image P. Also, the 
image output means 3 0 displays the calculated degree of certainty 
about malignancy D at a position in the vicinity of the abnormal 
pattern candidate PI . 

The abnormal pattern candidate detecting means 10 
stores an algorithm for abnormal pattern candidate detection 
processing utilizing an iris filter, wherein an image area, 
which is associated with a high degree of convergence of image 
density gradient vectors, is detected as a temporary candidate 
for an abnormal pattern (a tumor pattern) . The abnormal pattern 
candidate detecting means 10 also stores an algorithm for 
abnormal pattern candidate detection processing utilizing a 
morphological filter, wherein an image area, at which the image 
density changes in a range spatially narrower than a 
multi-structure element, is detected as a temporary candidate 
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for an abnormal pattern (a microcalcif ication pattern) . The 
abnormal pattern candidate detecting means 10 detects the 
temporary candidates for the abnormal patterns by use of the 
corresponding algorithms. Also, the abnormal pattern 
candidate detectingmeans 10 stores an algorithm for calculating 
a Mahalanobis distance, which reflects the similarity of an 
image pattern with respect to a malignant pattern or a benign 
pattern, in accordance with a plurality of feature measurements , 
which represent features of a certain image region. With 
respect to the detected temporary candidate, the abnormal 
pattern candidate detecting means 10 calculates a likelihood 
ratio Y of the aforesaid Mahalanobis distance as one of index 
values. The abnormal pattern candidate detecting means 10 
ultimately judges that a temporary candidate, which is 
associated with the likelihood ratio Y higher than a threshold 
value T2, is the abnormal pattern candidate PI to be detected. 

The malignancy certainty degree calculatingmeans 20 
calculates the degree of certainty about malignancy D with 
respect to the detected abnormal pattern candidate PI in 
accordance with the likelihood ratio Y of the Mahalanobis 
distance with respect to the detected abnormal pattern candidate 
PI, and also in accordance with the correlation between the 
likelihood ratio Y and the possibility of a pattern being a 
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malignant pattern, which has been obtained from the clinical 
results . In this embodiment , the relationship described below 
is employed as the correlation between the likelihood ratio 

Y and the possibility of a pattern being a malignant pattern. 
Specifically, the likelihood ratio Y of the Mahal anob is distance 
of each pattern is calculated with respect to each of a plurality 
of cases of diseases. Also, it is actually confirmed with 
pathological examinations, and the like, whether the state of 
the pattern associated with each value of the likelihood ratio 

Y is a malignant state, a benign state, or a normal state. In 
this manner, as illustrated in Figure 14, the relationship 
between the value of the likelihood ratio Y of the Mahalanobis 
distance and the proportion associated with each value of the 
likelihood ratio Y, which is occupied by each of the malignant 
state, the benign state, and the normal state, is obtained. 
The thus obtained relationship is employed as the correlation 
between the likelihood ratio Y and the possibility of a pattern 
being a malignant pattern, which has been obtained from the 
clinical results. Also, in this embodiment, the degree of 
certainty about malignancy D is calculated as the proportion 
associated with the value of the likelihood ratio Y of the 
Mahalanobis distance with respect to the calculation object 
region subjected to the calculation, which is occupied by the 
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malignant state. Specifically, as illustrated in Figure 14, 
in cases where the likelihood ratio Y of the Mahalanobis distance 
of the calculation object region is 1.7, the proportion 
associated with the likelihood ratio Y of 1.7, which is occupied 
by the malignant state, is 7 0%. Therefore, in such cases, the 
degree of certainty about malignancy D may be taken as 
being 70 (%) . 

How the first embodiment of the abnormal pattern 
candidate detection processing system of the present invention 
operates will be described hereinbelow. 

Firstly, the image signal P representing the 
mammogram is output from an external image forming modality, 
such as a computed tomography (CT) scanner or a computed 
radiography (CR) apparatus, into the abnormal pattern candidate 
detecting means 10 of the abnormal pattern candidate detection 
processing system 100. The abnormal pattern candidate 
detecting means 10 performs the abnormal pattern candidate 
detection processing on the received image signal P. The 
abnormal pattern candidate detection processing is performed 
in accordance with the aforesaid abnormal pattern candidate 
detection processing algorithms (i.e., the algorithm for the 
abnormal pattern candidate detection processing utilizing the 
iris filter and the algorithm for the abnormal pattern candidate 
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detection processing utilizing the morphological filter) . In 
this manner, the abnormal pattern candidate detecting means 
10 calculates index values (such as the degree of convergence 
of gradient vectors, the output value I of the iris filter, 
and the density of the microcalcif ication pattern candidates) 
K with respect to each area and region in the mammogram. Also, 
the abnormal pattern candidate detecting means 10 compares each 
of the index values K with a predetermined threshold value T, 
which corresponds to each of the index values K. In cases where 
each of the index values K is larger than the corresponding 
threshold value T, the pattern of the region associated with 
the index value K is taken as the temporary candidate . Further, 
the abnormal pattern candidate detecting means 10 performs 
discrimination processing on the temporary candidate and 
calculates the likelihood ratio Y of the Mahalanobis distance 
as an index value with respect to each of the temporary candidates 
A temporary candidate, which is associated with a likelihood 
ratio Y higher than the predetermined threshold value T2 , is 
extracted as the final abnormal pattern candidate PI . The 
abnormal pattern candidate detecting means 10 outputs 
information, which represents the position of the extracted 
abnormal pattern candidate PI, into the image output means 30. 
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The malignancy certainty degree calculating means 
2 0 calculates the degree of certainty about malignancy D with 
respect to the extracted abnormal pattern candidate PI in 
accordance with the likelihood ratio Y of the Mahalanobis 
distance, which is one of the index values and has been utilized 
for the detection of the extracted abnormal pattern candidate 
PI. The degree of certainty about malignance D is also 
calculated in accordance with the correlation between the 
likelihood ratio Y and the possibility of a pattern being a 
malignant pattern, which has been obtained from the clinical 
results. In cases where a plurality of abnormal pattern 
candidates PI, PI, ... have been detected, a plurality of degrees 
of certainty about malignancy D, D, ... corresponding to the 
abnormal pattern candidates PI, PI, ... are calculated. The 
malignancy certainty degree calculating means 2 0 outputs the 
information, which represents the calculated degree of 
certainty about malignancy D, into the image output means 30. 

The image output means 30 , into which the information 
representing the calculated degree of certainty about 
malignancy D is thus output, has already received the image 
signal P and the information, which represents the position 
of the detected abnormal pattern candidate PI . As illustrated 
in Figure 15, in accordance with the received image signal P, 
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the received information representing the position of the 
detected abnormal pattern candidate PI, and the received 
information representing the calculated degree of certainty 
about malignancy D, the image output means 30 displays the arrow 
mark, which indicates the abnormal pattern candidate PI in the 
image P, on the display screen. Also, the image output means 
3 0 displays the degree of certainty about malignancy D at a 
position in the vicinity of the arrow mark on the display screen . 

In Figure 15, mammograms P and P' of a set of the 
right and left breasts are displayed simultaneously such that 
the mammograms P and P' stand back to back with each other. 
However, the image layout is not limited to the layout described 
above. For example, the mammograms P and P ■ of a set of the 
right and left breasts may be displayed alternately. 

Also, besides the arrow mark indicating the abnormal 
pattern candidate PI, the mark for specifying the abnormal 
pattern candidate PI may be a closed curve or a rectangular 
frame, which surrounds the abnormal pattern candidate PI, or 
a number displayed in the vicinity of the abnormal pattern 
candidate PI. Further, as the information representing the 
degree of certainty about malignancy D, the kind of the mark 
may be altered in accordance with the degree of certainty about 
malignancy D. For example, the color, the thickness of the 
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line, the number of the lines, the figure of the number, or 
the like, may be altered in accordance with the degree of 
certainty about malignancy D. 

As described above, with the abnormal pattern 
candidate detection processing system 100, which is the first 
embodiment of the abnormal pattern candidate detection 
processing system of the present invention, the malignancy 
certainty degree calculating means 20 performs the calculation 
for finding the degree of certainty about malignancy D, which 
represents the level of possibility of a pattern being a 
malignant pattern, with respect to the abnormal pattern 
candidate PI detected by the abnormal pattern candidate 
detection processing. The calculation is perf ormed-perf ormed 
in accordance with the likelihood ratio Y of the Mahalanobis 
distance, which is one of the index values and concerns the 
judgment as to whether the candidate is to be detected as the 
abnormal pattern candidate PI in the abnormal pattern candidate 
detection processing. The calculation is also performed in 
accordance with the correlation between the likelihood ratio 
Y and the possibility of a pattern being a malignant pattern, 
which has been obtained from the clinical results. Also, the 
image output means 3 0 outputs and displays the information, 
which represents the calculated degree of certainty about 
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malignancy D with respect to the abnormal pattern candidate 
PI, together with the image P and the information for specifying 
the detected abnormal pattern candidate PI. Therefore, with 
the abnormal pattern candidate detection processing system 100 , 
the specific information useful for diagnosis is capable of 
being furnished to the operator who views the displayed medical 
image. Accordingly, the capability of the outputted image to 
serve as an effective tool in, particularly, the efficient and 
accurate diagnosis of an illness, is capable of being enhanced. 

A second embodiment, which is an embodiment of the 
second abnormal pattern candidate detection processing system 
of the present invention, will be described hereinbelow. 

Figure 16 is a block diagram showing a second 
embodiment of the abnormal pattern candidate detection 
processing system in accordance with the present invention. 
With reference to Figure 16, an abnormal pattern candidate 
detection processing system 200 comprises abnormal pattern 
candidate detecting means 210 for performing the abnormal 
pattern candidate detection processing on the received image 
signal P, which represents the mammogram, and thereby detecting 
the abnormal pattern candidate PI embedded in the image P. The 
abnormal pattern candidate detection processing system 2 00 also 
comprises region selecting means 215 for selecting an arbitrary 
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region in the image P . The abnormal pattern candidate detection 
processing system 2 00 further comprises malignancy certainty 
degree calculating means 220 for calculating the degree of 
certainty about malignancy D, which represents a level of 
possibility of a pattern being a malignant pattern, with respect 
to the pattern within a selected region P3 . The calculation 
of the degree of certainty about malignancy D is performed in 
accordance with an index value representing a feature of the 
pattern within the selected region P3 , and also in accordance 
with the correlation between the index value and the possibility 
of a pattern being a malignant pattern, which has been obtained 
from clinical results . The abnormal pattern candidate 
detection processing system 200 further comprises image output 
means 230. The image output means 230 displays an arrow mark 
at a posit ion to indicate the detected abnormal pattern candidate 
PI in the image P. Also, the image output means 23 0 displays 
the calculated degree of certainty about malignancy D at a 
position in the vicinity of the pattern within the selected 
region P3 . 

How the second embodiment of the abnormal pattern 
candidate detection processing system of the present invention 
operates will be described hereinbelow. 
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Firstly, the image signal P representing the 
mammogram is output from an external image forming modality, 
such as a computed tomography (CT) scanner or a computed 
radiography (CR) apparatus, into the abnormal pattern candidate 
detecting means 210 of the abnormal pattern candidate detection 
processing system 200. In the same manner as that for the 
abnormal pattern candidate detecting means 10 in the first 
embodiment described above, the abnormal pattern candidate 
detecting means 210 performs the abnormal pattern candidate 
detection processing on the received image signal P. The 
abnormal pattern candidate detection processing is performed 
in accordance with the abnormal pattern candidate detection 
processing algorithms described above. In this manner, the 
abnormal pattern candidate detecting means 210 extracts the 
abnormal pattern candidate PI. However, at this time, besides 
the calculation of the likelihood ratio Y of the Mahalanobis 
distance with respect to the temporary candidate, the abnormal 
pattern candidate detecting means 210 also performs the 
calculationof the likelihood ratioYof the Mahalanobis distance 
with respect to every area and every region in the image P 
regardless of whether the area and the region are the temporary 
candidates. The abnormal pattern candidate detecting means 
210 stores the information representing the thus calculated 
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likelihood ratios Y, Y, ... of the Mahalanobis distances. For 
example, the abnormal pattern candidate detecting means 210 
calculates an index value K with respect to a predetermined 
region in the image P, which has been set for each pixel in 
the image P. The thus calculated index value K is taken as 
the index value K corresponding to each of the pixels in the 
image P. Also, the abnormal pattern candidate detecting means 
210 calculates the likelihood ratio Yof the Mahalanobis distance 
which acts as an index value, from the index value K. The thus 
calculated likelihood ratio Y is taken as the likelihood ratio 
Y corresponding to each of the pixels in the image P. The 
abnormal pattern candidate detecting means 210 stores the 
information representing the calculated likelihood ratios Y, 
Y, ... Also, the abnormal pattern candidate detecting means 
210 outputs the information, which represents the position of 
the extracted abnormal pattern candidate PI, into the image 
output means 23 0. 

When an arbitrary region in the image P is selected 
with the region selecting means 215, the region selecting means 
215 outputs the information, which represents the position of 
the selected region P3 , into the malignancy certainty degree 
calculating means 220. 
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In accordance with the information representing the 
position of the selected region P3 , the malignancy certainty 
degree calculating means 220 reads the information representing 
the likelihood ratio Y of the Mahalanobis distance, which acts 
as the index value and corresponds to the selected region P3 , 
from the abnormal pattern candidate detecting means 210 . Also, 
the malignancy certainty degree calculating means 22 0 
calculates the degree of certainty about malignancy D with 
respect to the selected region P3 in accordance with the 
likelihood ratio Y of the Mahalanobis distance, which 
corresponds to the selected region P3 , and in accordance with 
the correlation between the likelihood ratio Y and the 
possibility of a pattern being a malignant pattern, which has 
been obtained from the clinical results. The malignancy 
certainty degree calculating means 220 outputs the information, 
which represents the position of the selected region P3 , and 
the information, which represents the calculated degree of 
certainty about malignancy D, into the image output means 230. 
At this time, the mean value or the maximum value of the values 
of the likelihood ratios Y, Y, ... of the Mahalanobis distances, 
which values correspond to all of the pixels lying within the 
selected region P3 , may be employed as the likelihood ratio 
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Y of the Mahalanobis distance , which corresponds to the selected 
region P3 . 

As illustrated in Figure 17, in accordance with the 
received image signal P, the received information representing 
the position of the detected abnormal pattern candidate PI, 
the received information representing the position of the 
selected region P3 , and the received information representing 
the calculated degree of certainty about malignancy D with 
respect to the selected region P3 , the image output means 230 
displays the arrow mark, which indicates the abnormal pattern 
candidate PI in the image P, on the display screen. Also, the 
image output means 23 0 displays the degree of certainty about 
malignancy D, which corresponds to the selected region P3 , at 
a position in the vicinity of the selected region P3 on the 
display screen. 

As described above, with the abnormal pattern 
candidate detection processing system 200, which is the second 
embodiment of the abnormal pattern candidate detection 
processing system of the present invention, the image output 
means 23 0 outputs and displays the image P and the information 
for specifying the detected abnormal pattern candidate PI . 
Also, the image output means 23 0 outputs and displays the 
information, which represents the degree of certainty about 
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malignancy D with respect to the selected region P3 , at the 
position in the vicinity of the selected region P3 in the image 
P. Therefore, besides the detected abnormal pattern candidate , 
the information representing the degree of certainty about 
malignancy with respect to an arbitrary image region is capable 
of being displayed and utilized for comparison. Accordingly, 
the capability of the outputted image serving as an effective 
tool in, particularly, the efficient and accurate diagnosis 
of an illness, is capable of being enhanced even further. 

A third embodiment, which is an embodiment of the 
third abnormal pattern candidate detection processing system 
of the present invention, will be described hereinbelow. 

Figure 18 is a block diagram showing a third embodiment 
of the abnormal pattern candidate detection processing system 
of the present invention. With reference to Figure 18, an 
abnormal pattern candidate detection processing system 300 
comprises abnormal pattern candidate detecting means 310 for 
performing the abnormal pattern candidate detection processing 
on the received image signal P, which represents the mammogram, 
and thereby detecting the abnormal pattern candidate PI embedded 
in the image P. The abnormal pattern candidate detection 
processing system 300 also comprises malignancy certainty 
degree calculating means 320 for calculating the degree of 
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certainty about malignancy D with respect to a predetermined 
region in the medical image, which has been set for each of 
the pixels in the image P, as the degree of certainty about 
malignancy D corresponding to each of the pixels in the image 
P . The calculation of the degree of certainty about malignancy 
D corresponding to each of the pixels in the image P is performed 
in accordance with an index value representing a feature of 
a pattern embedded in the predetermined region and in accordance 
with the correlation between the index value and the possibility 
of a pattern being a malignant pattern, which has been obtained 
from clinical results. The abnormal pattern candidate 
detection processing system 3 00 further comprises distribution 
image signal forming means 325 for forming a distribution image 
signal PD representing a distribution image PD, which represents 
a distribution of the degrees of certainty about malignancy 
D, D, ... in the image P, in accordance with the thus calculated 
degrees of certainty about malignancy D, D, . . . , each of which 
corresponds to one of the pixels. The abnormal pattern 
candidate detection processing system 300 still further 
comprises image output means 330. The image output means 330 
displays the image P and an arrow mark at a position to indicate 
the detected abnormal pattern candidate PI in the image P . Also, 
the image output means 33 0 displays the distribution image PD, 
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which represents the distribution of the degrees of certainty 
about malignancy D, D, ... in the image P. 

How the third embodiment of the abnormal pattern 
candidate detection processing system of the present invention 
operates will be described hereinbelow. 

Firstly, the image signal P representing the 
mammogram is output from an external image forming modality, 
such as a computed tomography (CT) scanner or a computed 
radiography (CR) apparatus, into the abnormal pattern candidate 
detecting means 310 of the abnormal pattern candidate detection 
processing system 300. In the same manner as that for the 
abnormal pattern candidate detecting means 10 in the first 
embodiment described above, the abnormal pattern candidate 
detecting means 310 performs the abnormal pattern candidate 
detection processing on the received image signal P. The 
abnormal pattern candidate detection processing is performed 
in accordance with the abnormal pattern candidate detection 
processing algorithms described above. In this manner, the 
abnormal pattern candidate detecting means 310 extracts the 
abnormal pattern candidate PI. However, at this time, the 
abnormal pattern candidate detecting means 310 calculates the 
index value K with respect to the predetermined region in the 
image P, which has been set for each of pixels in the image 
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P. The thus calculated index value K is taken as the index 
value K corresponding to each of the pixels in the image P. 
Also, the abnormal pattern candidate detecting means 310 
calculates the likelihood ratio Y of the Mahalanobis distance, 
which acts as an index value, from the index value K. The thus 
calculated likelihood ratio Y is taken as the likelihood ratio 
Y corresponding to each of the pixels in the image P. The 
abnormal pattern candidate detecting means 310 stores the 
information representing the calculated likelihood ratios Y, 
Y, ... Also, the abnormal pattern candidate detecting means 
310 outputs the information, which represents the position of 
the extracted abnormal pattern candidate PI, into the image 
output means 330. 

The malignancy certainty degree calculating 
means 32 0 reads the information representing the likelihood 
ratio Y of the Mahalanobis distance, which corresponds to each 
of the pixels in the image P, from the abnormal pattern candidate 
detecting means 310. Also, the malignancy certainty degree 
calculating means 320 calculates the degree of certainty about 
malignancy D with respect to each of the predetermined regions 
in the image P in accordance with the likelihood ratio Y of 
the Mahalanobis distance, which corresponds to each of the pixels 
in the image P, and in accordance with the correlation between 
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the likelihood ratio Y and the possibility of a pattern being 
a malignant pattern, which has been obtained from the clinical 
results. The thus calculated degree of certainty about 
malignancy D with respect to each predetermined region in the 
image P is taken as the degree of certainty about malignancy 
D corresponding to each of the pixels in the image P, which 
corresponds to the predetermined region. 

The distribution image signal forming means 3 25 forms 
the distribution image signal PD, which corresponds to the image 
P and represents the distribution image PD reflecting the degrees 
of certainty about malignancy, in accordance with the degrees 
of certainty about malignancy D, D, . . . , each of which 
corresponds to one of the pixels in the image P . The distribution 
image signal forming means 32 5 outputs the thus formed 
distribution image signal PD into the image output means 330. 

As illustrated in Figure 19, in accordance with the 
received image signal P, the received information representing 
the position of the detected abnormal pattern candidate PI, 
and the received distribution image signal PD representing the 
distribution image PD reflecting the degrees of certainty about 
malignancy, the image output means 330 displays the arrow mark, 
which indicates the abnormal pattern candidate PI in the image 
P, on the display screen. Also, the image output means 33 0 
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displays the distribution image PD, which corresponds to the 
image P and reflects the degrees of certainty about malignancy. 
The distribution image PD may be displayed by being superposed 
upon the image P and, for example, in a semi-transparent form. 
Alternatively, the image P and the distribution image PD may 
be displayed alternately on the display screen. 

As described above, with the abnormal pattern 
candidate detection processing system 300, which is the third 
embodiment of the abnormal pattern candidate detection 
processing system of the present invention, the image output 
means 330 outputs and displays the image P and the information 
for specifying the detected abnormal pattern candidate PI. 
Also, the image output means 33 0 displays the distribution image 
PD, which corresponds to the image P and reflects the degrees 
of certainty about malignancy. Therefore, the state of 
distribution of the degrees of certainty about malignancy with 
respect to the entire pattern of the image P is capable of being 
found. Accordingly, the width of the approach to diagnosis 
becomes broad, and the capability of the outputted image serving 
as anef f ective tool in, particularly, the efficient and accurate 
diagnosis of an illness, is capable of being enhanced even 
further . 
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The aforesaid first embodiment of the abnormal 
pattern candidate detection processing system in accordance 
with the present invention may be modified such that the 
information representing the degree of certainty about 
malignancy D is displayed only when an instruction to display 
the information representing the degree of certainty about 
malignancy D is given by the operator from a mouse device, or 
the like. With the modification described above, the viewing 
of the image area corresponding to the position for displaying 
the degree of certainty about malignancy D is not obstructed, 
and the degree of certainty about malignancy D is capable of 
being confirmed when necessary. 

The aforesaid first and second embodiments of the 
abnormal pattern candidate detection processing system in 
accordance with the present invention may be modified such that , 
as a different kind of the information representing the degree 
of certainty about malignancy D, a warning message, which is 
altered in accordance with the degree of certainty about 
malignancy D, may be displayed. For example, in cases where 
the degree of certainty about malignancy D is comparatively 
high, a warning message representing "HIGH MALIGNANCY" may be 
displayed at a position in the vicinity of the abnormal pattern 
candidate PI. Alternatively, in such cases, a warning message 
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representing "THERE IS A HIGHLY MALIGNANT PATTERN" may be 
displayed at a position spaced from the abnormal pattern 
candidate PI. In cases where the degree of certainty about 
malignancy D is comparatively low, a message representing "LOW 
MALIGNANCY" may be displayed at a position in the vicinity of 
the abnormal pattern candidate PI. Alternatively, in such 
cases, a message representing "THERE IS NOT A HIGHLY MALIGNANT 
PATTERN" may be displayed at a position spaced from the abnormal 
pattern candidate PI . 

Also, as illustrated in Figure 20, the relationship 
between the likelihood ratio Y of the Mahalanobis distance and 
the malignant pattern detection rate (i.e. , the proportion of 
the number of the malignant patterns, which are capable of being 
detected with the value of the likelihood ratio Y being taken 
as a threshold value, with respect to the total number of the 
malignant patterns) may be employed as the correlation between 
the index value and the possibility of apattern being a malignant 
pattern, which has been obtained from the clinical results. 
In such cases, the degree of certainty about malignancy may 
be calculated by the formula shown below. 



Degree of certainty about malignancy 

= 10 0 (%) - (proportion of number of 
malignant patterns associated with 
likelihood ratio) 
Further, the image inputted into the abnormal pattern 
candidate detection processing system is not limited to the 
mammogram and may be one of various images, such as a stomach 
X-ray image, which are to be subjected to the detection of the 
abnormal pattern candidate . 
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